Real-time Geometry-Aware Augmented Reality in Minimally Invasive Surgery

The potential of Augmented Reality (AR) technology to assist minimally invasive surgeries (MIS) lies in its computational performanceand accuracy in dealing with challenging MIS scenes. Even with the latest hardware and software technologies, achieving both real-timeand accurate augmented information overlay in MIS is still a formidable task. In this paper, we present a novel real-time AR frameworkfor MIS that achieves interactive geometric aware augmented reality in endoscopic surgery with stereo views. Our framework tracks themovement of the endoscopic camera and simultaneously reconstructs a dense geometric mesh of the MIS scene. The movement of the camerais predicted by minimising the re-projection error to achieve a fast tracking performance, while the 3D mesh is incrementally built by a densezero mean normalised cross correlation stereo matching method to improve the accuracy of the surface reconstruction. Our proposed systemdoes not require any prior template or pre-operative scan and can infer the geometric information intra-operatively in real-time. With thegeometric information available, our proposed AR framework is able to interactively add annotations, localisation of tumors and vessels,and measurement labeling with greater precision and accuracy compared with the state of the art approaches

Recent Developments and Future Challenges in Medical Mixed Reality

As AR technology matures, we have seen many applicationsemerge in entertainment, education and training. However, the useof AR is not yet common in medical practice, despite the great po-tential of this technology to help not only learning and training inmedicine, but also in assisting diagnosis and surgical guidance. Inthis paper, we present recent trends in the use of AR across all med-ical specialties and identify challenges that must be overcome tonarrow the gap between academic research and practical use of ARin medicine. A database of 1403 relevant research papers publishedover the last two decades has been reviewed by using a novel re-search trend analysis method based on text mining algorithm. Wesemantically identified 10 topics including varies of technologiesand applications based on the non-biased and in-personal cluster-ing results from the Latent Dirichlet Allocatio (LDA) model andanalysed the trend of each topic from 1995 to 2015. The statisticresults reveal a taxonomy that can best describes the developmentof the medical AR research during the two decades. And the trendanalysis provide a higher level of view of how the taxonomy haschanged and where the focus will goes. Finally, based on the valu-able results, we provide a insightful discussion to the current limi-tations, challenges and future directions in the field. Our objectiveis to aid researchers to focus on the application areas in medicalAR that are most needed, as well as providing medical practitioners with latest technology advancements

Self-supervised monocular image depth learning and confidence estimation.

We present a novel self-supervised framework for monocular image depth learning and confidence estimation. Our framework reduces the amount of ground truth annotation data required for training Convolutional Neural Networks (CNNs), which is often a challenging problem for the fast deployment of CNNs in many computer vision tasks. Our DepthNet adopts a novel fully differential patch-based cost function through the Zero-Mean Normalized Cross-Correlation (ZNCC) to take multi-scale patches as matching and learning strategies. This approach greatly increases the accuracy and robustness of the depth learning. Whilst the proposed patch-based cost function naturally provides a 0-to-1 confidence, it is then used to self-supervise the training of a parallel network for confidence map learning and estimation by exploiting the fact that ZNCC is a normalised measure of similarity which can be approximated as the confidence of the depth estimation. Therefore, the proposed corresponding confidence map learning and estimation operate in a self-supervised manner and is a parallel network to the DepthNet. Evaluation on the KITTI depth prediction evaluation dataset and Make3D dataset show that our method outperforms the state-of-the-art results

A family of integrable maps associated with the Volterra lattice

Recently Gubbiotti, Joshi, Tran and Viallet classified birational maps in four dimensions admitting two invariants (first integrals) with a particular degree structure, by considering recurrences of fourth order with a certain symmetry. The last three of the maps so obtained were shown to be Liouville integrable, in the sense of admitting a non-degenerate Poisson bracket with two first integrals in involution. Here we show how the first of these three Liouville integrable maps corresponds to genus 2 solutions of the infinite Volterra lattice, being the g = 2 case of a family of maps associated with the Stieltjes continued fraction expansion of a certain function on a hyperelliptic curve of genus g ⩾ 1. The continued fraction method provides explicit Hankel determinant formulae for tau functions of the solutions, together with an algebro-geometric description via a Lax representation for each member of the family, associating it with an algebraic completely integrable system. In particular, in the elliptic case (g = 1), as a byproduct we obtain Hankel determinant expressions for the solutions of the Somos-5 recurrence, but different to those previously derived by Chang, Hu and Xin. By applying contraction to the Stieltjes fraction, we recover integrable maps associated with Jacobi continued fractions on hyperelliptic curves, that one of us considered previously, as well as the Miura-type transformation between the Volterra and Toda lattices

Context-Aware Mixed Reality: A Learning-based Framework for Semantic-level Interaction.

Mixed Reality (MR) is a powerful interactive technology for new types of user experience. We present a semantic-based interactive MR framework that is beyond current geometry-based approaches, offering a step-change in generating high-level context-aware interactions. Our critical insight is that by building semantic understanding in MR, we can develop a system that not only greatly enhances user experience through object-specific behaviors, but also it paves the way for solving complex interaction design challenges. In this paper, our proposed framework generates semantic properties of the real-world environment through a dense scene reconstruction and deep image understanding scheme. We demonstrate our approach by developing a material-aware prototype system for context-aware physical interactions between the real and virtual objects. Quantitative and qualitative evaluation results show that the framework delivers accurate and consistent semantic information in an interactive MR environment, providing effective real-time semantic level interactions

Integrable maps in 4D and modified Volterra lattices

In recent work, we presented the construction of a family of difference equations associated with the Stieltjes continued fraction expansion of a certain function on a hyperelliptic curve of genus g. As well as proving that each such discrete system is an integrable map in the Liouville sense, we also showed it to be an algebraic completely integrable system. In the discrete setting, the latter means that the generic level set of the invariants is an affine part of an abelian variety, in this case the Jacobian of the hyperelliptic curve, and each iteration of the map corresponds to a translation by a fixed vector on the Jacobian. In addition, we demonstrated that, by combining the discrete integrable dynamics with the flow of one of the commuting Hamiltonian vector fields, these maps provide genus g algebro-geometric solutions of the infinite Volterra lattice, which justified naming them Volterra maps, denoted V_g. The original motivation behind our work was the fact that, in the particular case g=2, we could recover an example of an integrable symplectic map in four dimensions found by Gubbiotti, Joshi, Tran and Viallet, who classified birational maps in 4D admitting two invariants (first integrals) with a particular degree structure, by considering recurrences of fourth order with a certain symmetry. Hence, in this particular case, the map V_2 yields genus two solutions of the Volterra lattice. The purpose of this note is to point out how two of the other 4D integrable maps obtained in the classification of Gubbiotti et al. correspond to genus two solutions of two different forms of the modified Volterra lattice, being related via a Miura-type transformation to the g=2 Volterra map V_2. We dedicate this work to a dear friend and colleague, Decio Levi

Exploring drivers of within-field crop yield variation using a national precision yield network

1. While abiotic drivers of yields represent important limiting factors to crop productivity, the role of biotic drivers that could be directly managed by farmers (e.g. agri-environment schemes supporting key ecosystem services) remains poorly understood. Precision yield mapping provides an opportunity to understand the factors that limit agricultural yield through the interpretation of high-resolution cropping data. This has the potential to inform future precision agricultural management, such as the targeted application of agrochemicals, promoting increased sustainability in modern agricultural systems. 2. We used precision yield measurements from a network of 1174 fields in England (2006–2020) to identify drivers of within-field yield variation in winter wheat and oilseed rape. Potential drivers included climate, topography and landscape composition and configuration. We then explored relationships between in-field yield patterns and local landscape context, including the presence of features associated with ecosystem benefits. 3. Proximity to the field edge was associated with reduced yields in 85% of wheat and 87% of oilseed fields. This translating to an approximate reduction of 10% in wheat and 18% in oilseed yields lost due to field edge effects. 4. We found evidence that reduced yields at the field edges were associated with biotic features of the surrounding landscape, including the occurrence of semi-natural habitats. Specifically, agri-environment scheme (AES) presence increased the rate at which yields at field edges approach those of the field centres. This suggests that AES occurrence within a landscape (rather than field adjacent) may increase edge effects. However, these trends are unclear and suggest interactions between drivers and the spatial and temporal scale of investigation. 5. Synthesis and applications. While we found evidence of landscape context mitigating against field edge effects, these were counterintuitive. For example, AES at a landscape scale appeared to increase the severity of edge effects. This study highlights a lack of environmental data at sufficiently high spatiotemporal resolution to match that of precision agriculture data. This mismatch is hindering the effective integration of precision agriculture data in an environmental policy and/or management context and potentially leading to unnecessarily poorly informed decisions related to AES deployment. This may limit environmental and economic benefits

Mapping the ratio of agricultural inputs to yields reveals areas with potentially less sustainable farming

Fertilisers and pesticides are major sources of the environmental harm that results from farming, yet it remains difficult to target reductions in their impacts without compromising food production. We suggest that calculating the ratio of agrochemical inputs to yield can provide an indication of the potential sustainability of farmland, with those areas that have high input relative to yield being considered as less sustainable. Here we design an approach to characterise such Input to Yield Ratios (IYR) for four inputs that can be plausibly linked to environmental impacts: the cumulative risk resulting from pesticide exposure for honeybees and for earthworms, and the amount of nitrogen or phosphorus fertiliser applied per unit area. We capitalise on novel national-scale data to assess IYR for wheat farming across all of England. High-resolution spatial patterns of IYR differed among the four inputs, but hotspots, where all four IYRs were high, were in key agricultural regions not usually characterised as having low suitability for cropping. By scaling the magnitude of each input against crop yield, the IYR does not penalise areas of high yield with higher inputs (important for food production), or areas with low yields but which are achieved with low inputs (important as low impact areas). Instead, the IYR provides a globally applicable framework for evaluating the broad patterns of trade-offs between production and environmental risk, as an indicator of the potential for harm, over large scales. Its use can thus inform targeting to improve agricultural sustainability, or where one might switch to other land uses such as ecosystem restoration

Cosmological parameters from SDSS and WMAP

We measure cosmological parameters using the three-dimensional power spectrum P(k) from over 200,000 galaxies in the Sloan Digital Sky Survey (SDSS) in combination with WMAP and other data. Our results are consistent with a ``vanilla'' flat adiabatic Lambda-CDM model without tilt (n=1), running tilt, tensor modes or massive neutrinos. Adding SDSS information more than halves the WMAP-only error bars on some parameters, tightening 1 sigma constraints on the Hubble parameter from h~0.74+0.18-0.07 to h~0.70+0.04-0.03, on the matter density from Omega_m~0.25+/-0.10 to Omega_m~0.30+/-0.04 (1 sigma) and on neutrino masses from <11 eV to <0.6 eV (95%). SDSS helps even more when dropping prior assumptions about curvature, neutrinos, tensor modes and the equation of state. Our results are in substantial agreement with the joint analysis of WMAP and the 2dF Galaxy Redshift Survey, which is an impressive consistency check with independent redshift survey data and analysis techniques. In this paper, we place particular emphasis on clarifying the physical origin of the constraints, i.e., what we do and do not know when using different data sets and prior assumptions. For instance, dropping the assumption that space is perfectly flat, the WMAP-only constraint on the measured age of the Universe tightens from t0~16.3+2.3-1.8 Gyr to t0~14.1+1.0-0.9 Gyr by adding SDSS and SN Ia data. Including tensors, running tilt, neutrino mass and equation of state in the list of free parameters, many constraints are still quite weak, but future cosmological measurements from SDSS and other sources should allow these to be substantially tightened.Comment: Minor revisions to match accepted PRD version. SDSS data and ppt figures available at http://www.hep.upenn.edu/~max/sdsspars.htm